Streamlining Invoice Management with a Serverless Approach

Imagine me - staring at a pile of screenshots, calendar entries, and a calculator, dreading the monthly ritual of invoice preparation. Without any automation, my twice-monthly routine looked like this: screenshots scattered across my desktop (especially finding latest screenshots amongst the pile of all the screenshots), extracting data from screenshots (even though most values remain constant, but it’s thanos level threat when I took a leave or half), and a mental gymnastics on top of it.

The tipping point is when I mistakenly attach a screenshot from first month (the invoice which is already paid for), and another screenshot where I have added the leaves but that didn’t reflected in the final invoice, imagine that after a busy day where I can’t wait to log off, but all of these shenanigans are delaying my log off time.

The Invoice preparation saga

Let me paint you a picture of what my invoice preparation looked like:

1. Taking and saving multiple screenshots and squinting at timestamps
2. Cross-referencing with my calendar (and hoping I didn’t miss any days)
3. Adding everything into a spreadsheet (and checking everything is laser focused)
4. Performing currency conversion calculations (and double-checking because math is hard)
5. Generating multiple formats of the same invoice (because different people need same data differently, can’t blame them)
6. Review everything thrice because one mistake means the whole process starts over

This manual process was not just inefficient; it was a potential minefield of mistakes and frustration.

Enter the Serverless Solution

Why serverless? Because when you’re dealing with a task that runs twice a month, the last thing you want is to maintain a server that sits idle 99% of the time. AWS Lambda was perfect for this - it’s like having a helpful assistant who only shows up (and charges) when there’s actual work to be done.

System Design

Architecture Overview

View above diagram in full detail including all components and their place in the system in Mermaidjs

The system works like a well-oiled machine, with each component having a specific role:

1. S3 as the File Cabinet: Stores all our documents
2. Lambda as the Office Assistant: Processes files and generates invoices
3. DynamoDB as the Record Keeper: Maintains our data
4. Anthropic API as the Data Entry Specialist: Extracts information from images
5. SQS as the Task Manager: Ensures smooth workflow
6. HTMX as the User Interface: Keeps things simple and effective

System Workflow

View above diagram with more details including all components and full workflow in Mermaidjs

This workflow makes the invoice generation a breeze, instead of doing everything by hand. All I need is to upload files, give them some proper data before uploading, and wait for the system to do the rest. i.e. Using anthropic to process the images and extract the date and hours in the JSON format I want. Then use the format to generate the invoice, one more step to pass the conversion rate for the day and then the system generates all invoices in the click of the button. This helped me achieve more accuracy in the fraction of time.

The Technical Nitty-Gritty

Now, let’s get into the interesting technical decisions that make this system tick:

0. Why serverless and why not hosting your {Insert framework here} app (on docker)?

Well

1. I like serverless. It’s like having a well-trained assistant who only shows up (and bills) when needed. When you are minding your costs (I believe everyone does now a days), you will obviously go to the tech which you know is the cheapest and still the best.
2. Serverless excels at tasks like this - sporadic, burst workloads that don’t need 24/7 server uptime.
3. After working with both Express servers and serverless, I’ve found that unless my application stays in one machine and does the things locally, or I know lambda won’t fit in the workload (Like AI inferrence from a sequential call or working with large files in one go), serverless just makes more sense to me.

1. Why These Specific Services?

• S3 for Storage: Simple, reliable, and cost-effective. Plus, it triggers our Lambda functions automatically when new files arrive.
• Lambda for Processing: Pay only when we use it, perfect for our twice-monthly needs.
• DynamoDB: Fast, scalable, and works seamlessly with Lambda.
• HTMX: Because sometimes, simple is better than complex JavaScript frameworks. And hey, it doesn’t hurt to create something in an unknown tech to learn it.

2. Smart Optimizations

Here’s where things get interesting:

a) Lambda Layers for Code Sharing Instead of duplicating code across functions, I use Lambda layers to share common code and dependencies. Especially when serving HTMX pages, while each lambda generated their own page, but lot of assets (like javascript and themes) are common across all pages.

b) AWS CDK for Infrastructure While there are many infrastructure-as-code options, CDK with TypeScript gives us type safety and easier permission management. Which is a much welcome change over SAM or cloudformation templates.

c) S3 Metadata over Filenames Rather than playing the filename guessing game, I use S3 metadata to identify file types. It’s more reliable and makes the code cleaner.

Results: From Hours to Minutes

The end result? What used to take hours now takes about 5 minutes. But more importantly:

• No more squinting at screenshots
• No more calculator anxiety
• No more format juggling
• No more “did I miss anything?” stress

Lessons Learned

Building this system taught me several valuable lessons:

1. Serverless Shines in Periodic Tasks: When your task runs infrequently or can have some time toleration, serverless is often the perfect fit.
2. AI Can Handle the Boring Stuff: Let AI handle tedious tasks like extracting data from images.
3. Automation > Optimization: Sometimes, the best way to optimize a process is to automate it completely.

Wrapping Up

While this system is specifically tailored for my invoicing needs, the principles behind it are universal. Sometimes the best solution isn’t optimizing an existing process, but completely reimagining how it should work.

And remember, While I’m keeping this tool private, I believe sharing the architecture and lessons learned could help others automate their repetitive tasks. Maybe it’ll inspire others to see what’s possible with these tools.

May the force be with you.